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Objectives 


•  Provide  a  high-level  overview  of  the  origins,  foundations,  goals,  and 
scope  of  each  approach. 

•  Compare  and  contrast  support  for  quality  and  performance 
improvement 

•  Present  real-world  examples  of  a  multi-model  environment  that 
utilized  at  least  two  of  the  approaches 

•  Identifies  improvement  “truths”  that  apply  across  the  three 
approaches 
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About  the  Software  Engineering  Institute  (SEI) 


•  Federally  Financed  Research  and 
Development  Center  (FFRDC) 

•  Sponsored  by  the  Office  of  the 
Undersecretary  of  Defense  (OUSD) 
for  Acquisition,  Technology,  and 
Logistics  (AT&L) 

•  Created  in  1984  under  contract 
awarded  to  Carnegie  Mellon 
University 

•  Offices  in  Arlington,  VA;  Pittsburgh, 
PA;  and  Frankfurt,  Germany 

•  Steward  of  the  Capabiiity  Maturity 
Model  Integration 
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High  Level  Overview 
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Shared  History 


In  the  1930s,  Walter  Shewhart  began  work  in 
improvement  with  his  principles  of  statistical 
quality  control. 


Walter  Shewhart 

The  theories  of  process  management 
are  a  synthesis  of  the  concepts  and 
principles  refined  by: 

o  W.  Edwards  Deming 
o  Phillip  Crosby 

W.  Edwards  Deming  Phillip  Crosby 

o  Joseph  Juran 


Joseph  Juran 


Over  the  past  decades,  these  theories  have  informed  various  frameworks 
for  improvement  of  quality  and  performance. 


Source:  CMMI®  for  Development,  Version  1.3,  November  2010 

Source:  The  Capability  Maturity  Model:  Guidelines  for  Improving  the  Software  Process.  1995 
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Shared  Foundation  -  Plan-Do-Check-Act 


Originated  in  the  1920’s  with 
Shewhart 

Popularized  by  Deming,  who  later 
described  it  as  Plan-Do-Study-Act 

Now  considered  a  classical 
approach  to  continuous  quality 
improvement 

Shared  by  CMMI,  Baldrige,  and  ISO 
as  a  basic  foundational  principle 


http://www.iso.ora/iso/iso  cataloaue/manaaement  standards/manaaement  system  basics 


—  Three  Quality  Journeys 

Software  Engineering  Institute  CamegieMellon 

©2011  Carnegie  Mellon  University 


Goals  of  the  Capability  Maturity  Model  Integration 
(CMMI) 


CMMI  models  help  organizations  Improve  their  processes  and  focus  on 
activities  for  developing  quality  products  and  services  to  meet  the 
needs  of  customers  and  end  users. 


Scope  of  CMMI 

CMMI  has  been  described  as  a  "de  facto"  standard  with  77  countries 
reporting  appraisals.  An  estimated  1 .8  million  people  work  in  organizations 
that  have  had  at  least  one  appraisal  since  April  2002. 

The  CMMI  constellations  address  improvement  of  processes  for 
development  of  products,  acquisition,  and  services  for  industry  and 
government. 
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Origins  of  CMMi® 


Watts  Humphrey,  Ron  Radice,  and  others  extended  quality 
principles  to  software  in  their  work  at  IBM. 

Humphrey’s  book,  Managing  the  Software  Process,  provides 
basic  principles  and  concepts  on  which  many  of  the  Capability 
Maturity  Models®  (CMMs®)  are  based.) 

Beginning  in  1986,  the  SEI  and  MITRE  Corporation,  under  the 
direction  of  Watts  S.  Humphrey,  began  developing  a  process 
maturity  framework  to  help  organizations  improve  their  products. 


^  I 


Watts  Humphrey 


o  Response  to  government  request  for  method  to  assess  capability 
of  software  contractors. 

o  Software  process  assessment  method  and  software  capability 
evaluation  developed,  with  a  maturity  questionnaire  to  appraise 
software  process  maturity. 


Source:  CMMI®  for  Development,  Version  1.3,  November  2010 

Source:  The  Capability  Maturity  Model:  Guidelines  for  Improving  the  Software  Process.  1995 


_  Three  Quality  Journeys 

Software  Engineering  Institute  CamegieMellon  ssTcT3^%Tprf201T®® 

©2011  Carnegie  Mellon  University 


Progression  of  the  Capability  Maturity  Model 
Integration  (CMMI) 


•  Initial  release  of  the  CMM  reviewed  and  used  by  software  community  during 
1991  and  1992.  CMM  VI. 1  was  released  February  1993. 

•  Capability  Maturity  Model  (CMMI)  Integration  released  in  2000;  integrated 
Capability  Maturity  Model  for  Software,  v2.0  draft  C  (SW-CMM),  Systems 
Engineering  Capability  Model  (SECM),  and  Integrated  Product  Development 
Capability  Maturity  Model  (IPPD-CMM)  vO.98. 

•  CMMI  expanded  to  provide  multiple  constellations  -  CMMI-DEV  1 .2  (2006), 
CMMI  for  Acquisition  (2007),  and  CMMI  for  Services  (2009). 

•  Coordinated  cross-constellation  revision  and  upgrade  released  as  Version 
1.3  in  2010 


Software  Capability 
Maturity  Model  (SW- 
CMM) 


CMM  re-developed  DEV  ACQ 

as  integrated  CMMI  v  1 .2  v  1 .2 
DEV  1.1 


SVC 
V  1.2 


Development 

Acquisition 

Services 


Source:  CMMI®  for  Development,  Version  1.3,  November  2010 


Goals  of  Baldrige 

The  Baldrige  Program's  mission  is  to  improve  the  competitiveness  and 
performance  of  U.S.  organizations.  It  promotes  quality  awareness, 
recognizes  achievements  of  U.S.  companies,  and  publicizes 
successful  strategies. 

Scope  of  Baldrige 

The  Award  is  presented  annually  to  up  to  18  United  States  companies 
by  the  President  of  the  United  States.  There  have  been  95  award 
recipients  since  1988  across  six  organizational  categories: 

o  manufacturing  companies, 
o  service  companies, 
o  small  businesses, 
o  education  organizations, 
o  health  care  organizations,  and 
o  nonprofit  organizations. 


Source:  2011-2012  Criteria  for  Performance  Exceiience,  Maicoim  Baidrige  Nationai  Quaiity  Award 
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Origins  of  Baldrige  Award 

Malcolm  Baldrige  was  a  proponent  of  quality  management 
as  a  key  to  United  States  prosperity  and  long-term 
strength.  He  drafted  one  of  the  early  versions  of  a  national 
quality  act. 

At  a  White  House  Conference  in  1 983,  Baldrige,  along  with 
the  other  delegates,  strongly  believed  that  the  Issue  of 
quality  should  be  addressed  (Bogan  and  Hart).  They 
decided  that  they  wanted  something  similar  to  Japan’s 
Doming  Prize,  but  more  unique  to  the  United  States. 

Presented  in  1987  as  The  National  Quality  Improvement  Act,  it  was 
renamed  the  Malcolm  Baldrige  National  Quality  Award  Act  after  his 
untimely  death,  and  passed  by  Congress. 

Congress  voted  to  withdraw  funding  for  the  Baldrige  program  in  the  2012 
federal  budget. 

Source:  “Ten  Years  of  Business  Excellence  for  America,”  NIST  NOVEMBER  1998 

Source:  Bogan,  Christopher  E.,  Hart,  Christopher  W.  L.  (1992).  The  Baldrige.  New  York:  McGraw-Hill,  Inc. 
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Malcolm  Baldrige 


Progression  of  the  Baldrige  Criteria 


Dr.  Curt  Reimann,  the  first  director  of  the  Malcolm  Baldrige  National 
Quality  Program,  and  his  staff  at  the  National  Institute  of  Standards  and 
Technology  (NIST)  advanced  proposals  for  what  is  now  the  Baldrige 
Award  in  1987. 


Criteria  have  evolved  from  a  specific  focus  on  manufacturing  quality  to 
a  comprehensive  strategic  focus  on  overall  organizational  performance, 
competitiveness,  and  sustainability. 


It  is  considered  America’s  highest  honor  for  business  performance 
excellence. 


Baldrige 

Award 

established - 
manufacturing, 
service,  and 
small  business 
organizations; 


Baldrige  Award 
expanded  to 
education  and 
health  care 
organizations 


Nonprofit 

organizations, 

including 

government 

agencies, 

became 

eligible 


— 

New  name — 

Baldrige 

Performance 

Excellence 

Program 


Federal 
Funding  Cut 
for  FY12- 
Foundation 
to  Support 
Thru  2012 


Source:  Wikipedia,  http://en.  Wikipedia. org/wiki/Maicoim_Baidrige_Nationai_Quaiity_A ward 
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Goals  of  the  ISO  9000  Series 


The  ISO  9000  family  addresses  fulfillment  of  the  customer's  quality 
requirements  and  applicable  regulatory  requirements  while  aiming  to 
enhance  customer  satisfaction  and  achieve  continual  improvement. 

Scope  of  ISO  9000 

ISO  9000  Series  represents  an  international  consensus  on  good  quality 
management  practices.  ISO  9001 :2008  has  been  implemented  by  over  a 
million  organizations  in  176  countries. 


httD://www.iso.ora/iso/iso  cafe  management  svstems.htm 
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Origins  of  iSO  9000  Series 

The  organization’s  name,  which  is  officially  ISO,  comes  from 
the  Greek  word  "isos"  meaning  "equal"  (if  two  objects  meet  the 
same  standard,  they  should  be  equal) 

o  Organized  as  an  international  standards  body  in  1946 

o  Initially  addressed  standards  related  to  basic  mechanics,  such 
as  screw  threads,  ball  and  roller  bearings,  and  gears 

o  Membership  now  made  up  of  163  national  standards  institutes 
(31  Dec  2010) 

o  ISO  has  developed  over  18,500  International  Standards,  each  representing  a 
national  consensus  position  based  on  stakeholder  views  of  each  country 

As  part  of  the  evolution  in  the  certification  of  quality,  ISO  developed  the  ISO 
9000  series: 

o  Expanded  from  certification  of  quality  to  certification  of  the  management  of  quality 

o  Responded  to  companies’  desire  to  organize  their  own  approach  to  the  problem  of 
quality 

Source:  Raymond  Frontard,  Former  Director-General  of  AFNOR,  in  “Standards  Related  Activities” 
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Willie  Kuert,  last 
surviving  delegate  to 
1 946  conference 


Progression  of  the  ISO  9000  Series 


ISO  9000:1987  series  consisted  of  ISO  9000,  offering  fundamentals  and 
vocabulary;  and  three  9001  models  emphasizing  conformance  to  procedures  - 
ISO  9001  (new  products),  ISO  9002  (installation  and  services),  and  ISO  9003 
(final  inspection  and  test).  ISO  9000:1994  series  emphasized  quality  assurance 
via  preventive  actions.  ISO  9000:2000  series  combined  9001 , 9002,  and 
9003  into  9001 :2000  and  introduced  ISO  9004:2000.  Revisions  and 
clarifications  of  9001  and  9004  occurred  in  2008  and  2009,  respectively 

o  ISO  9001 :2000  -  eight  quality  management  principles,  process  performance 
metrics;  explicit  expectations  of  continual  process  improvement  and  tracking 
customer  satisfaction;  and  radical  change  to  emphasis  on  process  management; 
also  demands  involvement  by  upper  executives  in  order  to  integrate  quality  into  the 
business. 


o  ISO  9004:2000  standard  provided  method  for  sustaining  organizational  success 


o 

N 

o 

Initial  Release 

Released  - 

Released  - 

9001 :2000  amended 

o 

O) 

of 

ISO  9000:1994 

ISO  9000:2000 

to  9001:2008- 

o 

CO 

ISO  9000:1987 

amended 

ISO  9001 :2000 

ISO  9004:2000 

quality  management 
requirements 

> 


9004:2000  amended  to 
9004:2009  to  support 
management  of  sustained 
success  of  an  organization 
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Baldrige,  and  ISO  9000  Series 


Three  Quality  Journeys 
Charlene  Gross,  Jon  Gross 
SSTC  23-26  April  2012 

©2011  Carnegie  Mellon  University 


Carnegie  Mellon 


16 


Comparative  Timelines 


Initial  Release 
of 

ISO  9000:1987 


Software 

Capability 

Maturity 

Model 

(SW-CMM) 


r/- 


O)  ^ 


> 


Released  - 
ISO  9000:1994 
amended 


Original 
CMMI  DEV 
1.1  model, 
training,  & 
appraisal 
method 


CMMI- 

DEV 

V  1.2 


CMMI- 
ACQ 
V  1.2 


CMMI- 
SVC 
V  1.2 


Released  - 
ISO  9000 
ISO  9001 
ISO  9004 


2000 

2000 

2000 


9001 :2000 
amended  to 
9001:2008- 
quality 

management 

requirements 


CMMI  1.3 


Development 

Acquisition 

Services 


.9 


LU 

O 

Baldrige  j 

Baldrige  Award ; 

Nonprofit 

New  name — 

Award  i 

expanded  to  i 

organizations. 

Baldrige 

OC 

established-  I 

education  and  ! 

including 

Performance 

O 

manufacturing,  ■ 

health  care  ■ 

government 

Excellence 

< 

CQ 

service,  and  j 
small  business  | 
organizations;  I 

organizations  j 

w  1 

agencies, 
became  eligible 

Program 

9004:2000 
amended  to 
9004:2009  to 
support 

management  of 
sustained  success 
of  an  organization 


Federal 
Funding  Cut 
for  FY12- 
Foundation 
to  Support 
Thru  2012 
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High  Level  Comparison  -  Structure  and  Focus 


Comparison  of: 

CMMI  Model 

Baldrige  Criteria 

ISO  9000  Series 

Responsible 

Organization 

Software  Engineering 
Institute/Carnegie 
Mellon;  DOD  Federally 
Funded  Research  and 
Development  Center 

National  Institute  of 
Standards  and 
Technology  (NIST), 

US  Department  of 
Commerce 

Technical  Committee 
176  of  ISO - 
International 
Standards 
Organization 

Goal 

Improve  product 
quality  by  improving 
process  quality 

Improve 

competitiveness  and 
performance  of  US 
organizations 

Improve  quality  by 
improving 
management  of 
quality 

Applicability 

International 

United  States 

International 

Organizational 

types 

Generic  for  product 
development,  services, 
and  acquisition  in  any 
organization 

(1)  Manufacturing 

(2)  Service 

(3)  Small  business 

(4)  Education 

(5)  Health  care 

(6)  Nonprofit 

Generic  for  quality 
management  in  any 
organization 
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High  Level  Comparison  -  Structure  and  Focus 


Domain 

CMMI  Model 
Framework 

Baldrige  Criteria 
Framework 

ISO  9000 

High  level 
components 

Models,  training, 
appraisal  methodology 

Criteria, 

examination 

process 

Definitions  (9000), 

quality  management 

requirements 

(9001:2008), 

improvement 

sustainment 

(9004:2009) 

Structural 

Elements 

Structured  by  process 
areas  and  overarching 
generic  practices 

Structured  by 
Criteria  categories 

Structured  by  quality 
management  sections 

Achievement 

Structure 

Maturity  level  (ML  2-5) 
and/or  capability  level 
(CL  1  -  3)  rating 

Malcolm  Baldrige 
National  Quality 
Award 

9001 :2008  certification 
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Generic  Goals  and  Practices 

Generic  Goal  1  -  Achieve  Specific  Goals 

•  GP  1  -  Perform  specific  practices 
Generic  Goal  2  -  Institutionalize 
Managed  Process 

•  GP  2.1  -  Organization  Policy 

•  GP  2.2  -  Process  Plan 

•  GP  2.3  -  Resources 

•  GP  2.4  -  Responsibility 

•  GP  2.5 -Training 

•  GP  2.6  -  Product  Control 

•  GP  2.7  -  Stakeholder  Involvement 

•  GP  2.8  -  Process  Monitoring  and  Control 

•  GP  2.9  -  Objective  Evaluation 

•  GP  2.10  -  Management  Review  of  Status 
Generic  Goal  3  -  Institutionalize 

a  Defined  Process 

•  GP  3.1  -  Establish  a  Defined  Process 

•  GP  3.2  -  Collect  Process  Related 
Experiences 


Software  Engineering  Institute 


Core  Process  Areas  Maturity 

Level 

•  Configuration  Management  2 

•  Project  Monitoring  and  Control  2 

•  Project  Planning  2 

•  Process  and  Product  Ouality  2 

Assurance 

•  Requirements  Management  2 

•  Measurement  and  Analysis  2 

•  Decision  Analysis  and  Resolution  3 

•  Integrated  Project  Management  3 

•  Organizational  Process  Definition  3 

•  Organizational  Process  Focus  3 

•  Organizational  Training  3 

•  Risk  Management  3 

•  Organization  Process  Performance  4 

•  Ouantitative  Project  Management  4 

•  Causal  Analysis  and  Resolution  5 

•  Organizational  Performance  5 

Management 
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Baldrige  Focus 


Organizational  Profile: 
Environment,  Relationships, 
and  Strategic  Situation 


Strategic 

Planning 


1 

Leadership 


Customer 

Focus 


I  Leadership  Triad 


Workforce 

Focus 


Operational 

Focus 


7 

Results 


Results  Triad 


Measurement,  Analysis, 
and  Knowledge  Management 


Source:  2011-2012  Criteria  for  Performance  Exceiience,  Maicoim  Baidrige  Nationai  Quaiity  Award 
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ISO  9001 :2008  -  Focus 


[Product  realization  section  may  be  tailored 
to  meet  the  needs  of  your  organization] 


Software  Engineering  Institute  CamegieMellon 
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High  Level  Comparison  -  Evaluation 


Comparison 

of: 

CMMI  Model 

Baldrige  Criteria 

ISO  9000  Series 

Evaluation 

Method 

Appraisal  is  based 
on  model  goals  and 
practices 

Examination  is  based 
on  what  the 
organization  considers 
important,  as 
described  in  the 
organizational  profile 

Certification  is  based  on 
2001 :2008  compliance 

Evaluation 

framework 

Rating  by  maturity 
level  of  organization 
unit  (ML)  or  by 
capability  level  of 
process  area  (CL) 

Organizational 
examination  to  earn 
award 

Organizational  certification 

Lead  for 

evaluation 

Lead  Appraiser  of 
Standard  CMMI 

model 

Lead  of  Baldrige 
examiner  team 

External  Lead  auditor  from 
authorized  certification  body 

Frequency  of 
renewal 

Every  three  years 

Award  recipients 
ineligible  for  the  award 
again  for  five  years 

Initial  audit;  interim  audits  at 
least  every  12  months;  major 
re-certification  after  3  years 
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SCAMPI  Appraisal 


Appraisal  Team 


i 


Organizational 
Process  Suite 


Appraisal 

Requirements 

► 


The 
Process 


Actual 

Practice 


Lessons  Learned/ 
Improvements 

◄ - 


Findings, 

Recommendations 


Process 

Deployment 


Organization/ 

Projects 
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Baldrige  2012  Examination 


Organizational  Profile  that  is  completed  by  applicant  sets 
context  for  how  organization  operates  AND  for  the 
application  review.  Scoring  values  for  each  Baldrige  area 
are  shown  below: 


1  Leadership  (120  points) 

1 .1  Senior  Leadership  (70) 

1 .2  Governance  and  Societal  Responsibilities  (50) 

2  Strategic  Pianning  (85  points) 

2.1  Strategy  Development  (40) 

2.2  Strategy  Implementation  (45) 

3  Customer  Focus  (85  points) 

3.1  Voice  of  the  Customer  (45) 

3.2  Customer  Engagement  (40) 
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Baldrige  2012  Examination  (cont.) 

4  Measurement,  Analysis,  and  Knowledge  Management 
(90  points) 

4.1  Measurement,  Analysis,  and  Improvement  of  Organizational 
Performance  (45) 

4.2  Management  of  Information,  Knowledge,  and  Information  Technology 
(45) 

5  Workforce  Focus  (85  points) 

5.1  Workforce  Environment  (40) 

5.2  Workforce  Engagement  (45) 

6  Operations  Focus  (85  points) 

6.1  Work  Systems  (45) 

6.2  Work  Processes  (40) 

7  Results  (450  points) 

7.1  Product  and  Process  Outcomes  (120) 

7.2  Customer-Focused  Outcomes  (90) 

7.3  Workforce-Focused  Outcomes  (80) 

7.4  Leadership  and  Governance  Outcomes  (80) 

7.5  Financial  and  Market  Outcomes  (80) 
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ISO  9001 :2008  Implementation, 

Certification,  and  Ongoing  Maintenance 

•  From  the  start  of  the  project  to  the  initial  certification  audit 
certification  body  auditor  is  about  six  months 

o  Depends  on  resources,  size,  and  complexity 
o  Internal  audit  is  typically  scheduled  2  months  prior  to  each  external  audit 
to  allow  corrective  actions  prior  to  external  audit 

o  Instant  certification  not  possible  because  must  have  records  that 
demonstrate  use  of  the  quality  system 

•  Usually  have  a  surveillance  audit  by  external  auditor  six  months  after 
initial  certification 

•  Thereafter,  external  audits  are  conducted  at  least  every  1 2  months, 
and  sometimes  every  6  months  -  depending  on  the  business 

•  A  major  re-certification  audit  takes  place  after  three  years  to  renew 
the  ISO  9001  certificate 

Quality  Systems.  http://www.  qualitysystems.  com/support/display/qst/ISO+900 1 + Timeline+- 
+lmDlementation+and+Onaoina+Manaaement _ 
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Case  Study  #1  -  CMMI  +  Baldrige 

utilizing  both  CMMI  and  Baldrige  Criteria  for  Performance 

Excellence  approach. 

Example  Profile: 

o  US  Government  Organization  providing  heaith  care  administration 
and  services. 

o  improvement  history  in  the  IT  organization  spans  approximately  the 
last  nine  (9)  years 

o  The  iarger  enterprise  uses  Baldrige  Criteria  for  Heaith  Care 
Organizations. 

o  The  larger  enterprise  runs  an  internal  program  to  review  individual 
hospital  applications,  using  the  Baldrige  Criteria,  and  provides 
recognition,  based  on  internal  review  of  hospital  applications. 
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Case  Study  #1  -  CMMI  +  Baldrige  -  Major 
Change 

The  Information  Technology  portion  of  the  enterprise  started 
adoption  of  CMM.  IT  had  earlier  experience  with  Software  CMM. 

o  CMMI  model  based  improvement  and  adoption  was  started.  A 
series  of  SCAMPI  Class  B  and  C  appraisals  were  performed  to 
understand  process  gaps. 

o  Workforce  improvements  included  Program  Management  training 
and  encouragement  to  complete  PMI  Certified  Program 
Management. 

o  Improvement  action  plans  were  established  and  being  implemented. 

o  The  IT  portion  had  traditionally  been  improving,  but  was  separate 
from  the  enterprise’s  internal  program  applying  Baldrige  Criteria. 

Major  IT  issues  caused  dramatic  leadership  and  organizational 
changes  in  the  IT  portion  of  the  enterprise,  which  caused  the  early 
CMMI  improvement  actions  to  be  pushed  to  the  side. 
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Case  Study  #1  -  CMMI  +  Baldrige  -  Blended 
Approach 

New  IT  leadership  that  was  more  hospital-focused  began 
establishing  a  new  organizational  structure. 

o  Baldrige  used  to  organize  documentation  of  new  IT  organization. 
Analysis  of  performance  methods  informed  corrective  actions. 

Adoption  of  CMMI  for  Acquisition  and  Services  and  the  People 
CMM  were  restarted  for  guidance  in  operational  work  processes. 

o  Acquisition  and  Services  models  provided  best  approaches  to 
performing  acquisition  and  services  development  and  delivery  work. 

o  People  CMM  provided  best  approaches  to  develop  workforce. 

o  CMMI  Measurement  and  Analysis  and  Goal  Question  Indicator 
Measure  (GQIM)  supported  monitoring  product/services  environment. 

By  default,  a  multimodel  environment  was  established. 
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Case  Study  #1  -  CMMI  +  Baldrige  -  Blended 
Approach  Progress 

Multiple  completed  cycles  of  answering  the  Baldrige  Criteria 

o  Continued  evolution  of  documentation  to  performance. 

o  Set  of  measures  established  to  address  Baldrige  Results  Criteria, 
using  CMMi  Measurement  and  Analysis  and  GQIM  approaches. 

o  Neither  application  for  internal  Baldrige  Application  Review  Program 
submitted  nor  a  formai  SCAMPI  Appraisal  performed 

Future  direction? 

o  Use  of  Baldrige  Criteria  for  overaii  improvement  framework. 

o  Continued  multi-model  approach  with  CMMi  adoption  and  focused 
improvement  in  both  IT  services,  acquisition,  and  deveiopment;  and 
iT  workforce  human  capitai  deveiopment 

o  Major  poiitical  change  or  major  IT  system  failure  always  a  risk 
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Case  Study  #1  -  CMMI  +  Baldrige  -  Advantages 
and  Lessons 

•  Enterprise-wide  acceptance  and  internal  improvement  award  based 
on  Baldrige  Criteria  creates  a  climate  and  culture  for  improvement. 

•  Baldrige  approach  provided  IT  with  a  healthcare-focused 
framework  for  enterprise  improvement 

•  CMMI  informed  the  Baldrige  framework  with  details  on  best 
practices  for  IT  acquisition,  development,  and  delivery  of  products 
and  services 

•  CMMI  framework  traditionally  assumes  that  the  business  focus 
drives  improvement 

•  Baldrige  focuses  critical  attention  of  the  "Leadership,  Strategic 
Planning,  and  Customer  Focus”  and  on  Performance  Results 
(measurement). 

•  Baldrige  triads  lend  a  holistic  way  to  organize  the  improvement 
infrastructure  (working  groups). 
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Case  Study  #2  -  CMMI  +  ISO 


utilizing  both  CMMI  and  ISO  9000:2008  approach. 
Example  Profile: 

o  US  Government  Organization  providing  services  to  other 
Agencies 

o  Improvement  history  in  the  organization  spans  approximately  ten 
years 

o  Program  started  with  transition  of  new  leader  into  organization 
o  New  leader  brought  experience  of  improvement 
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Case  Study  #2  -  CMMI  +  ISO  -  Major  Change 

Enterprise  applied  CMMI  for  Development  to  acquisition  processes 
to  support  Program  Management  role  overseeing  Prime  Contractor. 
Change  agent  had  experience  with  Software  CMM  at  another 
Government  Agency. 

o  SCAMPI  A  completed,  followed  by  a  period  of  improvement. 

o  Emphasis  shifted  from  CMMI  for  Development  to  CMMI  for 
Acquisition,  with  series  of  SCAMPI  Class  B  and  C  appraisals  planned 
to  understand  process  gaps. 

o  Based  on  SCAMPI  B  results,  organization  began  considering  CMMI 
for  Services  as  a  future  SCAMPI  focus 

o  Management  demonstrated  fatigue  with  CMMI  implementation  and 
questioned  the  efficacy  of  implementing  CMMI  practices. 

Leadership  and  organizational  support  considered  moving  toward 
Baldrige  Criteria,  creating  a  possible  multimodel  environment. 
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Case  Study  #2  -  CMMI  +  ISO  -  Blended  Approach 

With  shifting  interests,  ISO  9000:2008  was  evaluated  as  a 
focused  approach  to  quality  management  going  forward. 

o  Provided  status  as  an  international  standard  and  was  independent 
from  previously  used  methods. 

o  Indicated  that  ISO  9000:2008  provided  roadmap  for  creating  a 
quaiity  management  system  that  addressed  issues  specific  to  this 
organization. 

o  In  addition,  ISO  9000:2008  exhibited  significant  synergy  with  both 
CMMI  and  Baldrige  with  emphasis  on  management  ownership, 
customer  requirements,  and  measurement 

o  Draft  policy,  objectives,  and  quality  management  manual  were 
presented  to  the  organization 
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Case  Study  #2  -  CMMI  +  ISO  -  Blended  Approach 
Progress 

Adoption  of  CMMI  for  Acquisition  and  Services  appear  to 
have  gained  ground 

o  CMMI  for  Acquisition  and  Services  models  provide  best  practices  to 
perform  acquisition,  IT  product,  and  services  work 

Future  direction? 

o  Organization  realignment  underway  and  possible  integration  with 
other  Government  units 

o  Organization  shifted  the  acquisition  strategy  and  integrated  to  a 
higher  level  organization 

o  Continued  use  of  CMMI  with  a  shift  in  focus  to  the  Services  model 
more  likely 
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Case  Study  #2  -  CM  Ml  +  ISO  -  Advantages  and 
Lessons 


•  Both  CMMI  and  ISO  9001 :2008  inform  implementation  of  a 
quality  management  systems 

•  Each  tool  provides  a  different  level  of  detail  on  managing 
quality 

•  The  CMMI  framework  traditionally  depends  on  and  assumes 
that  the  enterprise  has  established  and  maintained  the 
business  focus  that  drives  the  improvement  of  the 
development  and  delivery  of  products  and  services. 

•  ISO  9001 :2008  clearly  states  the  level  of  involvement  and 
ownership  that  must  be  accepted  by  senior  leadership. 
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Comparison  of  Scope 


Baldrige 

Performance  and 
Competitiveness  of 
the  organization. 


CMMI 

Product  and 
Services 
Development 


ISO 

9000:2008 

Quality 

Management 

System 
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Different? 


How  are  These  Choices 


Type  of  Organization 

o  Any  segment  of  business  or  government  - 


CMMI  or  ISO  (1^ 

o  Any  type  of  product  or  services  focusing  on  quality  management  - 
ISO 

o  Multiple  process  improvements  in  technical  products  or  services  - 
CMMI 

o  US  service  companies,  small  businesses,  education  organizations, 
health  care  organizations,  and  nonprofit  organizations  -  Baldrige 

Emphasis 

o  International  Standard  -  ISO 
o  International  Implementation  -  CMMI 
o  Organization  Performance  Excellence  -  Baldrige 


How  are  These 


Different? 


Outcome 

o  Internationally  Known  Rating  of  Process  Capability 
Organizational  Unit  Maturity  -  CMMI 
o  ISO  Certification  of  Compliance  with  International  Quality 
Management  Standard  -  ISO  9001 :2008 

o  Award  to  US  Organization  -  Malcolm  Baldrige  National  Quality 
Award 

Frequency  of  Renewal 

o  Required  renewal  every  three  years  to  retain  rating  or  certification 
-CMMI,  ISO 

o  Ineligible  to  apply  for  award  again  for  five  years  -  Baidrige 
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Where  are  the  Intersections?  Examples 


Active  involvement  by  senior  management 

Emphasis  on  process  management  -  Documenting, 
monitoring,  and  improving  processes 

Customer  Involvement 

Quality  assurance 

Sufficient  resources  including  personnel,  training,  the  facility,  and  work 
environment 

Measures  to  support  decision-making 

Records  to  support  claims  of  use 

If  you  pick  any  principle  of  quality  or  performance, 
each  journey  will  require  that 

mu  aadms  it  in  some  fasnion. 
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Which  Approach  Does  this  Represent? 


Steps  Toward  Mature  Processes 

An  Aid  for  Assessing  and  Scoring  Process  items 


(1)  Reacting  to  Problems 


Operations  tire  characterized  by  tictivities  rather  than  by 
processes,  and  they  tire  largely  responsive  to  immediate 
needs  or  problems.  Gotils  are  poorly  defined. 


(3)  Aligned  Approaches 


Operations  tire  characterized  by  processes  that  tire 
repeatable  and  regiiltirly  evaluated  for  improvement,  with 
learnings  shared  and  with  coordination  among  organiza¬ 
tional  units.  Processes  address  key  strategies  and  goals  of 
the  orgtinization. 


The  organization  is  at  the  beginning  stages  of  conducting 
operations  by  processes  with  repeatability,  evakiation 
tind  improvement,  and  some  early  coordination  among 
organizational  units.  Strategy  and  quantitative  goals  are 
being  defined. 


(4)  Integrated  Approaches 


Operations  are  characterized  by  processes  that  are  repeat- 
able  and  regularly  evaluated  for  change  and  improvement  in 
collaboration  with  other  affected  units.  Efficiencies  across 
units  are  sought  and  achieved  through  analysis,  innovation, 
and  the  sharing  of  information  and  knowledge.  Processes 
and  measures  track  progress  on  key  strategic  and  opera¬ 
tional  goals. 


Source:  2011-2012  Criteria  for  Performance  Exceiience,  Maicoim  Baidrige  Nationai  Quaiity  A\Nar6 
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Bottom  Li 


Utilizing  CMMI,  Baldrige  Criteria,  and/or  ISO  for 
quality  improvement  enriches  efforts  and 
benefits  each  individual  approach. 
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Three  Journeys  that  Support  Each  Other 


“The  Quality  of  the  Product  is  Highly  Influenced  by  the  Quality 

of  the  Process.  ” 


“Excellence  is  a  Journey,  Not  a  Destination” 

“Managing  the  Organization 's  Processes  so  that  They 
Consistently  Turn  Out  Product  that  Satisfies  Customers' 

Expectations” 


The  #1  Universal  Truth 


If  you  are  using  the  standard  as  a  marketing  gimmick 
(a  plaque  on  the  wall;  a  logo  on  a  website), 
or  just  because  a  customer  requires  it, 
then  the  standard  will  be 

a  mrden,  not  a  henefii " 

Anthony,  John.  http://www.  articlecube.  com/Article/lso-900 1  -Certification/1 4 777 1 0 
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Summary 


Provided  a  high-level  overview  of  the  origins,  foundations,  goals,  and 
scope  of  each  approach 

Compared  and  contrasted  CMMI,  Baldrige,  and  ISO  as  quality 
approaches 

Presented  two  real-world  examples  of  evolving  multi-model 
environments  that  utilized  combinations  of  CMMI,  Baldrige,  and/or  ISO 

Described  the  synergies  across  all  three  approaches 
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Contact  Information 


Presenter/Point  of  Contact 
Jon  L  Gross/Charlene  C  Gross 

Senior  Members  of  Technical  Staff 
Acquisition  Support  Program 
Telephone:  +1  412-268-5800 
Email:  iaross@sei.cmu.edu 
Email:  caross@sei.cmu.edu 

Web 

www.sei.cmu.edu 

www.sei.cmu.edu/contact.cfm 


U.S.  Mail 

Software  Engineering  Institute 
Customer  Relations 
4500  Fifth  Avenue 
Pittsburgh,  PA  1 521 3-261 2 
USA 

Customer  Relations 

Email:  info@sei.cmu.edu 
Telephone:  -i-l  412-268-5800 

SEI  Phone:  +1  412-268-5800 

SEI  Fax:  +1  412-268-6257 
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THIS  CARNEGIE  MELLON  UNIVERSITY  AND  SOFTWARE  ENGINEERING  INSTITUTE 
MATERIAL  IS  FURNISHED  ON  AN  “AS-IS"  BASIS.  CARNEGIE  MELLON  UNIVERSITY 
MAKES  NO  WARRANTIES  OF  ANY  KIND,  EITHER  EXPRESSED  OR  IMPLIED,  AS  TO  ANY 
MATTER  INCLUDING,  BUT  NOT  LIMITED  TO,  WARRANTY  OF  FITNESS  FOR  PURPOSE 
OR  MERCHANTABILITY,  EXCLUSIVITY,  OR  RESULTS  OBTAINED  FROM  USE  OF  THE 
MATERIAL.  CARNEGIE  MELLON  UNIVERSITY  DOES  NOT  MAKE  ANY  WARRANTY  OF  ANY 
KIND  WITH  RESPECT  TO  FREEDOM  FROM  PATENT,  TRADEMARK,  OR  COPYRIGHT 
INFRINGEMENT. 

Use  of  any  trademarks  in  this  presentation  is  not  intended  in  any  way  to  infringe  on  the  rights  of 
the  trademark  holder. 

This  Presentation  may  be  reproduced  in  its  entirety,  without  modification,  and  freely  distributed 
in  written  or  electronic  form  without  requesting  formal  permission.  Permission  is  required  for 
any  other  use.  Requests  for  permission  should  be  directed  to  the  Software  Engineering  Institute 
at  permission@sei.cmu.edu. 

This  work  was  created  in  the  performance  of  Federal  Government  Contract  Number  FA8721-05- 
C-0003  with  Carnegie  Mellon  University  for  the  operation  of  the  Software  Engineering  Institute, 
a  federally  funded  research  and  development  center.  The  Government  of  the  United  States  has 
a  royalty-free  government-purpose  license  to  use,  duplicate,  or  disclose  the  work,  in  whole  or  in 
part  and  in  any  manner,  and  to  have  or  permit  others  to  do  so,  for  government  purposes 
pursuant  to  the  copyright  license  under  the  clause  at  252.227-7013. 
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Acronyms 


CMMI  -  Capability  Maturity  Model  Integration 
CMMI-DEV  -  CMMI  for  Development 
PDCA  -  Plan-Do-Check-Act 

SCAMPI  -  Standard  CMMI  Appraisal  Method  for  Process  Improvement 
SEI  -  Software  Engineering  Institute 
U.S.  -  United  States 
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CMMI  Structure 


Process  Area 
(PA) 


Required 


Expected 


Specific  Goals 
(SG) 


Specific  Practices 
(SP) 


I 


i 


Example  Work 
Products 


Subpractices 


informative 


Generic  Goals 
(GG) 


I 


Generic  Practices 
(GP) 

1 


Subpractices 


Generic 

Practice 

Elaborations 


Purpose 

Introductory 

Related 

Statement 

Notes 

Process  Areas 
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Baldrige  Structure 


Organizational  Sustainability 


Communication  -  Engagomont  Organizational  Performance  Review 


r 


Leadership  Process 


Governance  -  Ethics 


Process  Improvement-Innovation 


Process  Control-Sustainability 


^j^l^hce^lrhp  iemehtation 


Work  Processes  Determination 


Supplier  and  Partner  Integration 


eadershi 

1 

m 

w 

Workforce  Engagement  Management 


Workforce  Engagement  Assessn>ent 


Recruit,  Hire,  Place,  and  Retain 


Career  Progression 


Learning  and  Development 


Workforce  Capability-Capacity 


Workforce  Ciintate  Improvement 


Workforce  Performance  Management 


Anaiysis 


Knowledge  Availability  -  Sharing 


Knowledge  Management 


Data,  Information,  Knowledge  Reliability 


TOTAL 

INTEGRATED 

BALDRIGE 

EXCELLENCE 

SYSTEM 


Societal  Responsibility 


Strategic 

Planning 

SYSTEM 


Strategy  Development  Process 
Action  Plan  Development 
Action  Plan  Deployment 
Strategic  Sourcing 
Risk  Assessment 
Strategic  Assessment 
Strategic  Resource  Commitment 
Resource  Allocation  -  Redirection 

I 


Product  Offerings  Determination 


Customer-Market  Segmentation 


Customer  Cu  iture  Bui  Iding 


Customer  Support 


Listenir>g-Compjajnt  Management 


Engagement-Satisfaction  Determination 


Review 


Benchmarking  and  Comparison 


Hardware  &  Software  Reliability  -  Security 


Performance,  Knowledge  Measures  Selection-Use 


j 


Source:  The  Baldrige  Management  Model,  htto'J/www. baldriae2 1 . com/Baldriae %20Model. html 
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ISO  9001 :2008  Structure 


Section  1 :  Scope 

Talks  about  the  standard  and  how  it  applies  to 
organizations 

Section  2:  Normative 
Reference 

References  another  document  that  should  be  used 
along  with  the  standard,  ISO  9000:2008,  Quality 
Management  Systems-Fundamentals  and  Vocabulary 

Section  3:  Terms/Definitions 

Gives  a  few  new  definitions 

Section  4:  General 
Requirements 

Gives  requirements  for  the  overall  Quality  Management 
System 

Section  5:  Management 
Responsibility 

Gives  requirements  for  the  overall  Quality  Management 
System 

Section  6:  Resource 
Management 

Gives  requirements  for  resources  including  personnel, 
training,  the  facility  and  work  environment 

Section  7:  Product 

Realization 

Gives  requirements  for  the  production  of  the  product  or 
service,  including  things  like  planning,  customer  related 
processes,  design,  purchasing  and  process  control 

Section  8:  Measurement, 
Analysis  and  Improvement 

Gives  requirements  on  monitoring  processes  and 
improving  those  processes 
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CMMI  Appraisal  Method 


Requirements  | 

Class  A  I 

Class  B  I 

Class  C 

Types  of  Objective 
Evidence  Gathered 

Documents 
and  interviews 

Documents 
and  interviews 

Documents  or 
interviews 

Ratings  Generated 

Goal  ratings  | 
required  \ 

Not  allowed 

Not  allowed 

Organizational  Unit 
Coverage 

Required  | 

Not  required  | 

Not  required 

Minimum  Team  Size  | 

4  ( 

2  I 

1 

Appraisal  Team  Leader 
Requirements 

Lead  Appraiser 

Person  trained 
and 

experienced 

Person  trained 
and 

experienced 

Source:  Appraisal  Requirements  for  CMMI,  Version  1.2  (ARC) 
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Baldrige  Application 


Fixed  annual  application  review  schedule  (Spring  through 
Fall).  The  identity  of  all  applicant  organizations  remains 
confidential,  unless  they  receive  the  award. 


Application  Process 

o  Two-step  process: 

—  Submit  an  eligibility  certification  package. 

—  Submit  an  award  application  package  -  5  pages  of  profile  information  and 
50  pages  covering  7  criteria  categories. 

Application  Review 

o  Independent  and  Consensus  Review  conducted  by  a  team  from  the 
Board  of  Examiners. 

o  Site  Visit  Review  for  higher-scoring  organizations, 
o  Final  reviews  and  award  recommendations  by  Panel  of  Judges. 
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Baldrige  Application  Review  (cont.) 

Application  Feedback 

o  Examiner  review  team  provides  2  scores 

o  How  well  institutionalized  for  a  category 

o  How  effective  measurement  program  for  a  category 

o  Written  feedback  findings  for  the  applicant  to  consider  for 
improvement 

Award  Recipients 

o  Award  recipients  may  publicize  and  advertise  their  awards 

o  Recipients  are  expected  to  share  information  about  their 
successful  performance  strategies  with  other  U.S.  organizations 
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Required  ISO  9001 :2008  Documentation 

1.  Document  Control  (per  ISO  9001  clause  4.2.3)  -  Documents 

required  by  the  quality  management  system  shall  be  controlled.  Records 
are  a  special  type  of  document  and  shall  be  controlled  according  to  the 
requirements  given  in  4.2.4.  A  documented  procedure  shall  be  established 
to  define  the  controls  needed 


2.  Record  Control  (per  ISO  9001  clause  4.2.4)  -  Records  established  to 
provide  evidence  of  conformity  to  requirements  and  of  the  effective 
operation  of  the  quality  management  system  shall  be  controlled.  The 
organization  shall  establish  a  documented  procedure  to  define  the  controls 
needed. 

3.  Internal  Audit  (per  ISO  9001  clause  8.2.2)  -  The  organization  shall  conduct 
internal  audits  at  planned  intervals  to  determine  whether  the  quality 
management  system: 

o  conforms  to  the  planned  arrangements  (see  7.1 ),  to  the  requirements 
of  this  International  Standard  and  to  the  quality  management  system 
requirements  established  by  the  organization,  and 

o  is  effectively  implemented  and  maintained. 
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Required  ISO  9001 :2008  Documentation 
(cont.) 

4.  Control  of  Non-Conformities  (per  ISO  9001  clause  8.3)  -  The 
organization  shall  ensure  that  product  which  does  not  conform  to  product 
requirements  is  identified  and  controlled  to  prevent  its  unintended  use  or 
delivery.  A  documented  procedure  shall  be  established  to  define  the 
controls  and  related  responsibilities  and  authorities  for  dealing  with 
nonconforming  product. 

5.  Corrective  Action  (per  ISO  9001  clause  8.5.2)  -  The  organization  shall 
take  action  to  eliminate  the  causes  of  nonconformities  in  order  to  prevent 
recurrence.  Corrective  actions  shall  be  appropriate  to  the  effects  of  the 
nonconformities  encountered.  A  documented  procedure  shall  be 
established  to  define  requirements 

6.  Preventive  Action  (per  ISO  9001  clause  8.5.3)  -  The  organization  shall 
determine  action  to  eliminate  the  causes  of  potential  nonconformities  in 
order  to  prevent  their  occurrence.  Preventive  actions  shall  be  appropriate 
to  the  effects  of  the  potential  problems.  A  documented  procedure  shall  be 
established  to  define  requirements. 

ISO  9001  Training.  Understanding  ISO  9001:2008.  Ask  Art  Solutions.  2006-2010 
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